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I. INTRODUCTION AND BACKGROUND 


The Model Output Statistics (MOS) technique involves the 
processing of atmospheric parameters output from numerical 
weather prediction models (predictors), along with observed 
data, to produce forecast algorithms of meteorological 
parameters (predictands). The predictands are either 
operationally important parameters not forecast by numerical 
models (e.g.,visibility, cloud amount, ceiling) or model 
output parameters whose predictive skills are improved 
(e.g.,surface wind, temperature) due to partial correction 
of numerical model bias and/or scale. 

The National Weather Service (NWS) uses a linear, 
least-squares regression model to generate empirical 
forecast equations. This MOS technique has demonstrated 
operationally usable skill in forecasting numerous weather 
elements at land locations throughout the world [Best ana 
Pryor, 1983]. Both the United States Air Force and Navy 
have made limited use of the NWS model for selected land 
areas around the world. The Navy has attempted to forecast 
Open-ocean fog and visibility using linear regression 
equations, with the resultant skill levels exceeding 
persistence, climatology and those of the NWS as well. 
However, these limited experiments produced results 


considered only marginally useful for operational situations 
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meldaneer,1979; Yavorsky, 1980; Selsor, 1980; Koziara, et 
al, 1983; Renard and Thompson, 1984]. Undoubtedly, this 
performance level is due, in part, to the lack of 
'calibrated' fog and visibility observations. At sea, 
weather observers lack the reference points necessary to 
accurately estimate the visibility. 

Because of the potential for success demonstrated by the 
above cited experiments, the Navy began development of an 
MOS program in the spring of 1983 to forecast operationally 
Nu Ptant air/ocean parameters over all ocean areas in both 
hemispheres. Horizontal visibility was selected as the 
first parameter to be investigated due to its importance to 
the mariner. Because linear regression techniques over land 
areas (NWS, 1960--date) and the North Pacific Ocean (Navy, 
late 1970's) demonstrated considerably less-than-perfect 
results, other statistical methods were proposed to 
determine if a better one could be found. 

Preisendorfer (1983 a,b,c) proposed three strategies, 
two based on maximum probability and one based on natural 
regression. Lowe (1984a) proposed innovative threshold 
techniques to be applied with the linear regression 
approach. All of these methods were developed, applied and 
tested on North Pacific and North Atlantic Ocean areas by 
Karl (1984) and on additional North Atlantic Ocean areas by 


Pemmaizio) (1904¢a) in their investigations of visibility. 
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Wooster (1984) applied the same techniques to cloud amount 
and ceiling height parameters. 

This study presents a Principal Discriminant Method 
(PDM) of statistical analysis as developed for the MOS 
problem by Preisendorfer (1984). Significance testing 
methods proposed by Mr. Paul Lowe, Naval Environmental 
Prediction Research ከ ከ and investigated by Diunizio 
(1984b), were also utilized. These results are compared 
with results obtained from the aforementioned methodologies. 

In the following discussion, a sufficient number of 
terms and symbols are defined to allow readers without 
strong statistical backgrounds to understand the results. 
However, for a proper understanding of the Preisendorfer 
(1984) methodology, readers are encouraged to examine 
Appendix A for a detailed discussion. Details on the 
significance testing [Diunizio, 1984bl are found in Apr PE 


B. 


Conversation, and unpublished notes. 
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ll. OBJECTIVES AND APPROACH 


The objective of this study is to determine if the 


Principal Discriminant Method (PDM), applied to discrete 


values of model output and derived parameters, can improve 


upon the forecasting of horizontal marine atmospheric 


visibility when compared to the Preisendorfer natural 


regression and maximum probability approaches. The PDM 


approach is outlined as follows: 


as 


lom 


define visibility groups, categorized in a way which 
relates most closely to operational use at sea. 


develop and apply the Preisendorfer (1984) PDM to 
three North Atlantic Ocean physically homogeneous 
EE ስ ስ በስ ት ቓዳሙጢችጢሸንብ ሞስከስስችውችስ 2፡51 May through 7 July 1983 
Navy Operational Global Atmospheric Prediction System 
(NOGAPS) predictor data. 


compare and contrast the individual results with those 
Preisendorfer statistical methodologies previously 
Stony Karl (1984) and Diunizio (1984a). 


Based on a. to c. above, present an interim 
recommendation for an optimal statistical approach 
to forecasting horizontal visibility in the North 
Atlantic Ocean as a function of prediction time and 
homogeneous area. 
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III. DATA 


A. VISIBILITY CBSERVATIONS ANDAS NOIA OS 

Horizontal visibility observations taken from seagoing 
platforms are reported as values of ten standardized World 
Meteorological Organization (WMO) synoptic weather codes 
(Appendix C). These codes range in value from 90, which 
corresponds to visibility less than 50 m, to 99, which 
corresponds to visibility equal to or greater than 50 km. 
Human observational error and inexactness in measuring 
visibility at sea necessitate a reduction of visibility 
classification categories for prediction purposes. 

1. Three-Category Case 

Initially, a three visibility category 


classification scheme was consideres 


Visibility Category Synoptic Code Visibility Range 
I 90-94 <2 km 
B 95-96 >2 km to <10 km 
Ji i E 210 km 


The above scheme is the same as that used by Karl (1984) 
and Diunizio (1984a); it is based upon the following at-sea 
operational criteria followed by the U. S. Navy. 

1, 10 km (5 n mi)--U.S. Navy aircraft carrier at-sea 
flight recovery operations change from visual (VFR) to 


controlled (IFR) approach guidelines [Department of the 
Navy, 1979]. 
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2. 2 km (1 n mi)--the sounding of reduced visibility 
signals for all vessels operating in international waters. 
The term "reduced visibility" is not specifically defined in 
the International Regulations for Preventing Collisions at 
Sea, 1972. The distance of 1 n mi is generally considered 
to be the governing operational distance. 


2. Two-Category Case 

In the past [Renard and Thompson, 1984], forecasting 
skill for category II has proved to be minimal. In the 
preliminary work for this study, it was noted that the 
predictor means of all three category subsets, as a function 
of associated predictand values, were not always well 
separated. Without good separation, a good statistical 
forecast is not possible regardless of the method used. It 
was noted however, that even though not all three means were 
well separated, at least two of the means were well 
separated from each other. This finding suggested that a 
two-category case might be better supported by the data. If 
the two-category case showed better data support than the 
three-category case, then enhanced results might be 
expected. To test this hypothesis, two different 
two-category data sets were created for experimentation. 


The two cases are: 


Case X 

Visibility Category Synoptic Code Visibility Range 
IX 90-95 <4 km 
TPX 96-99 24 km 


e 


Case Y 


Visibility Category Synoptic Code Visibility Range 
jn 90-94 <2 km 
Ti 95-99 22 am 


B. NORTH ATLANTIC OCEAN DATA 
1. Area 
The North Atlantic Ocean, iron O to 80 N latitude, 
was divided into homogeneous oceanic areas by Lowe (1984b), 
using a statistical cluster analysis technique. The 
homogeneous areas evaluated in this study are identified as 
areas 2, 3W and 4 which represent areas of moderate, 
frequent and sparse occurrences of poor visibility, 
respectively (Fig. 1). 
2 Time Louis 
Data from mid-May 1983 to mid-July 1983 were 
combined to form a more extensive data set, hereafter 
referred to as FATJUNE 1983. The FATJUNE period was 
selected as the initial data set for statistical 
experimentation because of the climatologically high 
frequency of occurrence of poor visibility observations for 
many areas of the North Atlantic Ocean during this period. 


Only the 1200 GMT synoptic ship report data, corresponding 


“However, NOGAPS predictor data for the period 15 May--7 
July 1983 only were available for the study. 
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momdayilileht conditions, were used in this preliminary study 
ot the method. 

For the purpose of this study, TAU-00 generally 
represents six-hour model forecast fields. However, 
temperature, geopotential height and wind are model 
initialization fields. TAU-24 and TAU-48 are defined as 
24-h and 48-h model forecast fields. All of the above are 
valid at 1200GMT. TAU-00, TAU-24 and TAU-48 model output 
parameters (predictors) are employed in the O0O-h, 24-h and 
48-h forecast schemes, respectively. Summaries of the number 
of observations in each visibility category of the dependent 
and independent data sets, as a function of homogeneous area 
and prediction time for FATJUNE 1983, are contained in 
Tables I-IV. 

3. Synoptic Weather Reports 

All synoptic visibility observations (predictand 
data) for this study were provided by the Naval Oceanography 
Command Detachment (NOCD), Asheville, North Carolina which 
[momeo-located with the National Climatic Data Center (NCDC). 
The observations which contained systematic observer error 
or were obviously erroneous, as determined from the data 
quality indicators provided with the data, were deleted from 
the working data sets. 

4, Predictor Parameters 
PO F, titty-rour TAU-24 and fifty-four 


TAU-48 model output predictors (MOP's) were provided by the 
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Fleet Numerical Oceanography Center (FNOC), Monterey, 
California. The parameters are generated by their current 
operational atmospheric prediction model, NOGAPS. All MOP's 
were interpolated from model grid coordinates to synoptic 
ship report positions using a linear interpolation scheme. 
In addition to the initial group of MOP's, thirteen derived 
parameters representing calculated quantities, such as 
parameter gradients and products, were included as potential 
predictors. Of the available predictor parameters, fifteen 
were eliminated from consideration because 1) the MOP lacked 
a physical linkage to the visibility predictand, arno c TEES 
lack of significant digits (lost during the transfer of Whe 
FNOC data to the main computer center mass storage system) 
rendered the particular MOP useless. A list of all TAU-00, 
TAU-24 and TAU-48 MOP's available to the experiments are 


included in Appendix E. 


C. DATA SETS 
lu. Standarizacion 
NOGAPS analysis/forecast parameters are output in a 

large variety of units/scales. To eliminate the effect of 
different units of the various predictors on the Principal 
Discriminant Method (particularly the part using principal 
component analysis), the data were standardized before the 
method was applied. Given X... X. members in each 


predictor group, the standardized members Hunnen are 


E 


given by 
x 


7 ስሉ” 
i S 


where 


n 
RAD Xj the mean 
Se 


Ihe 


n 
S Li id , the unbiased estimate of the 
H7 ^j-21 standard deviation. 
In this way all units were removed, the data centered at O, 
and the variance of each of the data sets became 1. 
2.  Dependent/Independent Data Sets 
Since FATJUNE 1983 was the only data set available 
for this study, the data were divided into two groups. 
Approximately two-thirds of the data became the dependent 
set upon which the model was based. This set is also 
referred to as the tralning set. The remaining one-third of 
the data became the independent set on which the model was 
tested. This set is also referred to as the testing set. 
To insure that no biases existed in the sets, each 
training-testing set pair was created by use of a uniform 
random number generator. The given data sets were randomly 
split and then checked to insure they represented the 
initial population mean within a 95% confidence interval. 
Once created, these sets were used consistently throughout 


all model runs. 
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An 


Jo PROCEDURES 


TERMS AND SYMBOLS 


The following terms and symbols are used throughout the 


remainder of this thesis and are briefly defined here to 


assist the reader. For more definitive mathematical 


expressions of potential errors, consult Appendix A. 


Mathematical expressions for class errors, threat scores and 


adjusted class errors may be found in Appendix D. 


1. 


AO0--the estimated probability (based on actual 
predictions using the testing set) of a zero-class 
visibility category forecast errore: E 
visibility category I 1s forecast, iiS use 
observed). 


Ai--the estimated probability (based on actual 
predictions using the testing set) of a one-class 
Visibility category forecast enn Mee 
visibility category I is forecast and category 

II is observed). 


A2--the estimated probability (based on actual 
predictions using the testing set) of a two-class 
visibility category forecast error F n pm 
visibility category I is forecast and category III 
is observed). 


PAO--the estimated probability (based on the 
training set) of a zero-class visibility category 
forecast error. 


PA1--the estimated probability (based on the 
training set) of a one-class visibility category 
error. (PA2 is defined similarly.) 


Potential skill scores--(PAO,PA1 above) may be 
interpreted as follows. Randomly partition a data 
set (such as FATJUNE 1983) many times into 
training-testing set pairs. Fit probability 
distributions to the category subsets of the 
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DL. 


12. 


training set as described in PDM. Then produce 
PAO, PAI values (using the training set) and 
actual AO, Al values (using the testing set). 
Repeat this for all the training-testing set pairs. 
Take the average of all PAQ values and all AO 
values: In the limit of a sufficiently large 
number of partitions of the data set, these 
averages will tend to agree. Similarly for PAl, 
Mit and PAZ, A2. 


Correlation coefficient--a numerical measure of 

the relationship between one predictor and another. 
The value of the correlation coefficient ranges 
from -1 for negative correlation to +1 for positive 
correlation. The larger the absolute value of the 
correlation coefficient, the more closely are the 
predictors correlated. 


P-value--the result of a two-sided significance 
test on separate variance t-test statistics. This 
gives a measure of the separation of the data into 
different visibility categories. 


TSl--threat score for visibility category I 
computed from a contingency table. 


Maximum probability strategy--choosing forecast 
Visibility category based upon the highest 
conditional probability of the predictand 
categories for a given a predictor interval. 


MAXPROB I--designation of a maximum probability 
strategy in which ties of the highest conditional 
probabilities in a predictor interval are resolved 
by the generation of a random number 


MAXPROB II--designation of a maximum probability 
strategy in which ties of the highest conditional 
probabilities for a given predictor interval are 
resolved by assigning the lowest visibility 
category, of those ties, as the forecast category. 


Natural regression strategy--choosing forecast 
Visibility categories based upon the statistical 
average of the conditional probabilities of 
Visibility for a given predictor interval. 


Functional dependence. This is a measure of the 
stochastic dependence of one predictor upon another. 
Functional dependence is an estimate of the 

probability that one of the predictors will change 


e) 


1193 


D 


1:95 
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when the other changes. High functional dependence 
values between one already selected predictor and 
another potential predictor indicates that little 
additional information beyond the selected predictor 
is possible. The specific derivation and 
mathematical description of the concept of 
"functional dependence" is discussed in greater 
depth by Preisendorfer (1983c). 


Root-sum-squared functional dependence. The 
functional dependence of a predictor on all 
predictors already included in the developmental 
model. It is equal to the square-root of the sum 
of the squares of the individual functional 
dependence values. 


AAO--adjusted AO. A contingency table statistic 
which removes the influence of the most frequent 
visibility category in a set of data (similar to 
a normalized value). 


CE--class error parameter defined as AO -* 2A1 
used as the primary aid in identifying the first 
predictor in the Preisendorfer (1983a,b,c) PR models. 


PP--the potential predictability of visibility 
by any given predictor. 


B. COMPUTER PROGRAMS 


Four computer programs were developed to test the 


proposed Preisendorfer (1984) Principal Discriminant Method 


(PDM) methodology. The programs are on file in the 


Department of Meteorology, Naval Postgraduate School, 


Monterey, Calli tom aco ar 


1. 


ሯ 


A program to standardize the data and create 
training and testing sets for homogeneous areas, 
depending on whether the two-or three-category 
strategy was in use. 


A program to compute correlation coefficients between 
chosen and unchosen predictors, sorting them from 
low to high values. 
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Swe program to compute PAO, PAI, AO and Al values 
for each predictor and to check the PAO values for 
Significance against chance. 
!. A program to compute PAO, PATI, AO and Al values 
for two or more predictors using binary decomposition. 
This program also computes contingency tables 
and threat scores. 
freee nc lLoENDORFER PDM METHODOLOGY 
Determination of the First Predictor 
Selecting the first predictor is a two-step process. 
The first step involves computing the initial statistics 
(PAO, PA1) for each predictor. Secondly, based on output 
from BMDP Statistical Software program P7D [University of 
California, 1983],the average P-value for each predictor is 
computed and these values are ranked from low to high. The 
low values indicate better separability of the category 
populations. Therefore, the first predictor chosen is the 
one with the smallest averaged P-value. If more than one 
predictor shares the same low P-value, then of those 
predictors, the one with the highest PAO value is selected 
Ene first predictor. 
2. Choosing the Second and Subsequent Predictors 
The prospective second predictor in the model is 
determined from its correlation coefficient with the already 
chosen first predictor. The prospective second predictor 
has the smallest absolute value of the correlation 


coefficient. Whether it will ultimately be chosen as the 


second predictor depends on the following: 
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a. PAO has increased, and 
b. PA1 has decreased or remained constant, and 


c. the averaged P-value is significant, i.e., less 
tham D E 


If the prospective predictor cannot meet these criteria, 
then the next least correlated predictor is tried until all 
predictors have been exhausted. 

This process is repeated for the multi-predictor 
stage until the model is complete. 

3. Terminating the Selection of Predictors 

Model development continues until any one of he 

following Tour condivwons we emer: 
a. no more predictors remain to be considered, or 


b. PAO and/or P-scores are no longer significant with 
respect to the null hypothesis, or 


C. criteria required to add additional predictors 
cannot be met. 


Once the model development is complete, actual 
zero-and one-class errors (A0,A1) are computed using the 
independent data set. The resulting PAO, PAl, AO and Al 
values provide the measurement statistics on which the 


usefulness of the model is based. 


D. PREISENDORFER (PR) MODEL 

This model represents the application of the 
Preisendorfer (1983a,b,c) methodology (PR) explored by 
Karl (1984) and Diunizio (19842). Karl's study provides 


specific details on the method and readers interested in a 
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more thorough presentation may consult it. This discussion 
is presented as a prelude to comparing results of the PR 
model to the Preisendorfer (1984) PDM model of this study. 

As with the PDM model, the PR model utilizes NOGAPS 
model output and derived parameters as potential predictors 
in constructing a developmental model, based upon the 
dependent training data set, which provides the structure by 
amen the model is tested and evaluated. (However, as 
applied by Karl and Diunizio, the data sets were not formed 
randomly nor were the means of the sets constrained to be 
representative of the entire population from which they were 
drawn. Instead, the visibility category groups were 
constrained to show similar percentages, for both the 
independent and dependent data sets.) The range of values 
of these predictors is partitioned into discretized equally 
populous predictor intervals ("cells") and conditional 
probabilities of the predictand are calculated according to 
the three previously defined VISCAT's. There are three 
separate strategies for determining the VISCAT to be 
identified with each predictor value. These strategies are 
MAXPROB I and MAXPROB II based on maximum probability, and a 
natural regression approach. 

The sizes of the equally populous predictor intervals 
are varied from four to ten. An optimal first predictor is 
selected, which meets (in order) one of the following 


requirements: 


መ 


a. the lowest CE value of all the potential predictors, 
OT 


b. the highest PP value of all potential predictors. 
After selecting a first predictor for each of the 
equally populous intervals, the corresponding VISCAT I, II 

and III threat and EE EEN 
dependent and independent data sets from the MAXPROB II 
strategy. Then the optimal equally populous predictor 
interval is selected such that it is the smallest interval 
to maximize the dependent data set's adjusted AO and 
independent data set's adjusted VISCAT I threat score 
(Appendix D). 

Next, a functional dependence test-of the first 
predictor against the remaining potential predictors is run. 
Subsequent predictors are selected only if: 


a. the AO value increased over that at the preceding 
level, and 


b. the selected predictor must have the lowest 
functional dependence and root-sum-square functional 
dependence of all the remaining potential predictors. 
After completing the predictor selection stage, Monte 
Carlo significance testing is performed to see if the 
results are significant compared to random chance. 
Functional dependence/root-sum-square functional dependence, 
AO and Al statistics are calculated for 100 randomly 
generated sets to determine the 5 and 96 percentile points 


of Al and AO, denoted as 'A1(05)', 'AO(96)', respectively. 


The developmental model results are considered to be 
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Eu-cnificant if: 
a. AO is greater than or equal to A0(96), and 
DN Ads less than or equal to A1(05), and 
c. the functional dependence value for a selected 
predictor is less than functional dependence 96 
percentile level FD(96) (determined by the Monte 
Carlo procedure, above). 

Model development continues until the fifth predictor 
level when computer storage limitations preclude further 
addition of predictors. Once complete, contingency tables 
of forecast versus observed visibility category are 


constructed for both dependent and independent data sets. 


Threat and skill scores are computed and compared. 


Zoe LHe PDM VS. PR METHODS 

The PDM method and the PR methods can be shown to be 
equivalent in the discrete setting; it is in the 
non-discrete setting that they differ by virtue of fitting 
one or the other with analytic versions of discrete 
probabilities. The MAXPROB approaches make a prediction 
based on the probability distribution of the categories for 
a given predictor value, whereas the PDM method 
discriminates between the probabilities of the categories in 
a predictor space. In the PDM method, analytic functions 
are fitted to the category subsets of predictor space and 
comparisons are made between these probabilities at each 
given predictor value. Thus, more continuous information is 


available when the data are sparse in this method than with 
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the MAXPROB approach (although the latter, too, may be 
fitted with analytic probability models). Both of these 
methods should have an advantage over more traditional 
linear regression techniques whenever the data shows 
nonlinear rather than linear trends over the predictor 
Space, Since these methods would tend to follow the curve of 
the data, instead of trying to fit a straight line (or 
hyperplane) to them. The more predictor categories that are 
used and the more nonlinear the predictand/predictor 
relation, the greater. is the anticipated advantage of PDM 


over linear regression. 


30 


V. RESULTS 


The results of the Principal Discriminant Method (PDM) 
experiments, as outlined in Chapter IV and Appendix A, are 
presented herein. They are arranged by oceanic homogeneous 
area and model output period. Fig. 1 displays the 
individual oceanic homogeneous areas for FATJUNE 1983. 
Tables I through IV identify the number of observations in 
Gach visibility category by prediction interval (i.e., TAU) 
and homogeneous area. 

The results are further clarified by the corresponding 
figures in Appendix G, which provide comparisons of PAO and 
dependent and independent AO scores versus the number of 
predictors chosen for that particular data set. The models 
for each set terminated due to established model constraints 
and not due to computer system storage restrictions. Note 
that dependent AO scores were not available at the first 
predictor level due to programming time and constraints. 
Future experiments could include this information. Thus, 
the dependent AO data start with the second predictor. The 
chosen predictors are listed in the order of selection. 
Contingency tables resulting after the selection of the 
final predictor are included for both dependent and 
independent sets. In general, independent AO (testing) 


scores are lower than the dependent (training) AO scores. 
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Even though the training and testing sets are representative 
of the same population, their points are scattered 
differently. This difference, in general, leads to a 
decrease in the AO scores from the dependent (training) set 
to the independent (testing) set. However, in a number of 
cases, the independent AO score is higher than the PAO score 
at the first predictor level. Likewise, the dependent AO 
score is higher than the PAO score at the second predictor 
level for some cases. Although, on average, one would 
expect the reverse to be true, the sone of the individual 
test scores could occasionally lead to higher AO scores than 
PAO scores. The steady decline of AO scores for the first 
few ds is also a common occurrence. While the PAO 
score continues its steady ascent (as required by the method 
to justify the addition of the next predictor) the AO scores 
shows more erratic behavior, exhibiting the instability of 
the method. However, when the criteria test was changed, as 
will be described later, the resulting AO values show a 
closer relationship to the PAO seores and hence greater 
stability. Stability is desired in a model or else its 
forecasts are of little value. To determine exactly why the 
method is stable or unstable, carefully controlled 
experiments would have to be performed with artificial data 
Seto. 

When comparing the results of the PDM model to the 


maximum probability and natural regression strategies of the 
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PR models, it was noted that the PR models provided higher 
scores in almost every case for all scoring techniques. 
This difference may be due to the composition of the data 
sets themselves, the separation of the data into 
training-testing pairs, some aspect of the methodology ora 
programming error. However, without conducting experiments 
Smmeareciully constructed artificial data sets it would be 
impossible at this point to state a conclusive reason for 
the difference. One PDM experiment which was conducted at 
the end of the research did give comparable results to the 
PR methods and will be discussed in more detail later as 
will any other exceptions to the general finding stated 
above. Specific numerical values from the work of Karl 
(1984) and Diunizio (1984a), along with the corresponding 


ENRDEresults, are presented in Table V. 


A. NORTH ATLANTIC OCEAN, AREA 2 

Area 2 encompasses a geographic region extending from 
the southeastern tip of Newfoundland, across the North 
Atlantic Ocean to the eastern coast of England, 
north/northeast to include most of Iceland, and back to the 
Canadian coast north of Newfoundland (Fig. 1). 

1. Area 2, TAU-00 

Results for this case are shown in Fig. 6. Five 

predictors were selected. The dependent AO score rises 


slightly between predictors two and three, and then roughly 


p» 


parallels the independent AO score. The independent AO 
score does not show an increase over its initial value until 
the addition of the fifth predictor. The PDM model 
outperforms the PR model in the following scores (Table V): 
a. TS2 scores for both dependent and independent sets 
are higher than for either MAXPROB I or MAXPROB II, 
thus showing better skill in forecasting VISCAT II. 


b. The TS12 score is higher for the dependent set 
than either MAXPROB I or MAXPROB II. 


2. Area 2, TAU-24 
A variety of experiments were performed on this 
case. In addition to the standard application of the PDM 
techniques afforded the other cases, two additional 
three-category experiments and a two-category experiment 
were performed, as detailed in paragraphs a, b, and c below. 
a. Set Composition Experiments 
To determine the effect of the random 
composition of training/testing set pairs on the results, 
three distinct sets (2(A,B,C)) were created. 
Sets 2(A,B,C) follow a similar pattern for the 
PAO scores, except that sets 2(B) and 2(C) could not support 
more than five predictors, while the model for set 2(A) 
finally terminated with the seventh predictor (Fig. 7). The 
first five predictors were the same in all three cases. The 
dependent AO scores (Fig. 8) follow a different pattern in 
each case (one (A) declining to predictor four and then 


increasing and decreasing once more, one (B) declining 
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Breadily, and one (C) declining to predictor three and then 
increasing and decreasing once more) which is fairly well 
paralleled by the independent AO scores (Fig. 9). Ideally, 
the curves in Figs. 8 and 9 should be as closely spaced as 
those in Fig. 7. Presumably, these scattered curves are 
showing giving information about the noise inherent in the 
observed visibility data sets. Also, they may indicate that 
PAO, PAI must be redefined so that they may more 
realistically anticipate these scatterings of the AO, Al 
scores. Separate figures for each set are found in Figs. 
wee il and 12. 

The PDM vs., PR results found for area 2, TAU-00 
hold true at TAU-24 also. 

b. Criteria Experiments 

To determine the effect of altering the criteria 
for splitting data swarms in predictor space during the 
decomposition phase, two methods were tried. The first 
entailed changing the critical A value from A(96) to A(98). 
The second eliminated Monte Carlo methods entirely and 
created a new value, A', where A' is the ratio of the 
largest eigenvalue (associated with the data swarm's 
covariance matrix) to the average of the remaining 
eigenvalues. For the A' experiments, the set was split 
if እ'>ረ. 

The criteria tests, which were all performed on set 


Beye show that changing the critical value from A(96) 
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to A(98) leaves the PAO pattern basically unchanged. (Note 
that the same seven predictors were used in each of the 
criteria experiments.) The pattern for the A' curve is much 
different, exhibiting a slower rise to the sixth predictor 
and then a large jump at the end, surpassing the results of 
the other criteria tests (Fig. 13). The major difference is 
in the behavior of the dependent and independent AO scores 
(Figs. 14 and 15). The A(96) test curve in Pig. 14% Show m 
sharp decline in the dependent AO score to the fourth 
predictor, an even sharper rise at the fifth predictor and 
decline thereafter. These scores are mirrored by the lower 
scoring independent AO's in Fig. 15. Both sets of scores 
are considerably less than the PAO scores of Fig. 13. 

The A(98) test gives a more stable version of the same 
pattern. Unlike the first two tests, the A' test produces 
dependent AO scores in Fig. 14 quite similar to Fig. 13's 
PAO score through predictor six, with independent scores 
following a roughly similar pattern without a major loss in 
zero-error skill. These results show much greater stability 
than for any other experiments conducted and thus show the 
most promise for a continued investigation of the PDM 
method. The dependent and independent AO scores for the A 
version of PDM are comparable to those of the PR methods. 
Some of the skill scores are higher for PDM and some for PR. 
The point here is that "fine tuning” the criteria cut-off 


(from A'>2.0 to some other value) could result in superior 
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scores overall. The A(98) and A' cases are treated 
8 ከከ. dually in Figs. 16 and 17. Once again, in comparing 
the curves in Figs. 16 and 17, we see that the A' version of 
PDM produces much more stable and somewhat higher scores 
mee the A(98) version. 

C. Two-Category Experiments 

The two-category cases (Chapter III.A.2) provide 
quite different final results when compared with each other, 
even though the general pattern was not much different 
between Case X and Case Y(A). (Note that there are three 
versions of Case Y, i.e., Y(A,B,C). All comparisons -between 
ees and Y were done with the Y(A) data set.) The 
results for Case X are shown in Fig. 18. The model 
terminated at the fifth predictor level with the same five 
predictors as for the other Area 2, TAU-24 cases. Both 
dependent and independent AO scores decline through 
predictor number three, then slowly increase to a level much 
below their respective initial AO scores. 

The three Case Y sets exhibit the same 
similarities in the PAO scores as the three-category cases 
Mere.) 19). Again, the AO scores (Figs. 20 and 21) tend to 
show a pattern of decline followed by increasing scores. 
However, in these two-category cases, the independent AO 
scores are very close to the dependent AO scores and they 


are considerably higher than for the the three-category 


PI 


cases. Individual results for each case are presented in 
Figs. 22, 23 anda 
Case X does not show AO scores appreciably 
higher than the AO scores from the three-category case. 
This might indicate that the data do not support the Case X 
VISCAT divisions any more than for the three-category case. 
However, Case Y does show significantly higher AO scores 
than the three-category case. This is more in line with the 
expected result; expected since it ought to be easier to 
forecast for two categories than for three under any 
circumstances. This result seems to indicate that the Case 
Y VISCAT divisions are supported by the available data, 
l.e., that it may be more feasible for on-board observers to 
discern between less than or greater than 2 km visibility, 
than less than or greater than 4 km visibility. 
3. Area 2, TAU-48 

Results for this case are shown in Fig. 25. Four 
predictors were selected. The dependent AO scores stay 
virtually constant for all predictors. The independent AO 
scores show a continuous decline with each additional 
predictor. The PDM shows higher dependent threat scores and 


an independent TS2 score when compared to MAXPROB I. 


B. NORTH ATLANTIC OCEAN E 
Area 3W borders the United States' eastern seaboard from 


the vicinity of Cape Charles, Virginia to the southeastern 
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tip of Newfoundland. The area encompasses a large portion 
of the Georges Banks region and extends to approximately TEN 
W longitude (Fig. 1). 
l. Area 3W, TAU-00 
Results for this case are shown in Fig. 26. Five 
predictors were chosen. Once again, both dependent and 
independent AO scores decline until the addition of the 
fron predictor, at which point they surpass their initial 
values. 
2. Area 3W, TAU-24 
The results for this case are shown in Fig. 27. 
Three predictors were selected. In this case, the 
independent AO score increase with the addition of each 
predictor, while the dependent AO score decreases. This 
pattern is not seen in any other case. 
3. Area 3W, TAU-48 
The results for this case are shown in Fig. 28. 
Seven predictors were chosen. The dependent and independent 
AO scores decline until the addition of the fifth predictor 
where they reach their maximums. The sixth predictor shows 
another decline and the seventh an increase. The 
independent TS2 score in the PDM model equals that of the PR 


model. 


e. 


C. NORTH ATLANTICAO SPAN AREATA 
Area 4 encompasses a broad region of the North Atlantic 
Ocean which is generally to the south of area 2 and east and 
southeast of area 3W. This area's southern border reaches 
to the northeastern tip of Portugal and extends northward 
through the English Channel to encompass the southern 
portion. bebe ee 
l|. Ares v! ann 
The results for this case are shown in Fig. 29. 
Only two predictors were chosen for this model. The 
independent AO score declines after the addition of the 
second predictor. The dependent AO score shows no trend 
since only one value was available. 
2. Area 4, TAU-24 
The results for this case are shown in Fig. 30. 
Four predictors were chosen. The dependent and independent 
AO scores declined until the addition of the fourth 
predictor. At that point, they are larger than at their 
initial predictor stage. The dependent and independent 
results are almost identical which is a rare result. The 
PDM model shows higher scores for all dependent threat 
scores compared to MAXPROB I and the independent TS2 score 
from MAXPROB I. 
3. Area 4, TAU-48 
The results for this case are shown in Fig. 31. Two 


predictors were chosen. The independent AO score increases 
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from the first to the second predictor. No trend is 


available for the dependent AO score. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

The primary objective of this study is to evaluate the 
Principal Discriminant Method (PDM) [Preisendorfer, 1984], 
to compare those results to the maximum probability and 
natural regression schemes [Preisendorfer, 1983a,b,c] 
examined by Karl (1984) and Diunizio (1984a), and to propose 
a viable statistical forecasting scheme suitable for 
eventual employment in an operational U. S. Navy marine 
Visibility MOS forecasting system. In general, the PDM 
model, using the A(96) criteria for decomposing sets, was 
outperformed in all measures of effectiveness by all of the 
PR schemes. 

However, the version of the PDM model which used the A 
criterion for splitting predictor cateto Setomaa Ing 
decomposition showed very promising results (cf., 2b in V.A. 
and Table V). The AO/A1 scores for both dependent and 
independent sets were very close to those of the PR models. 
Perhaps, this is because the A' criterion is a better judge 
of the geometry of the data sets than the Monte Carlo A(96) 
criterion. The result is that the information contained in 
the data set is more readily available in the A' method than 


for the other predictor space category splitting methods. 
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B. 


RECOMMENDATIONS 


The following recommendations are offered to future 


pesearchers: 


P 


The decision criteria for splitting data swarms 

in the decomposition phase need further examination. 
Indeed, it is the novel use of principal component 
analysis for this purpose that distinguishes the 
present discriminant method from other such methods 
in the literature. The A' criterion appears to be 
c cou ከሠ rhehny direcvilon™ (note 2b in*V.A.). 
Further research should center on determining the 
best value against which to test the A value, or 
still other ways of splitting the overly-elongated 
category subsets of predictor space. 


Create carefully controlled artificial data sets 

on which to apply all of the Preisendorfer models 
(MAXPROB, natural regression, PDM) to determine where 
and why they break down or excel. Also, using the 
same artificial data, simultaneously test regression, 
especially linear, along with the various threshold 
models. 


Remove from further consideration the A(96) and A(98) 
critical score criteria in the decomposition phase 
Of the model: 


Test the PDM model, using the entire FATJUNE 1983 
data set as the training set and the entire FATJUNE 
1984 data set as the testing set, or vice versa. 


Use winter data for a set of experiments to 
determine if the results are similar to that of 
the summer season. 


Use a night-time data set (0000 GMT) in the 


North Atlantic area to test the expected deterioration 
of all schemes relative to daytime conditions. 
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ነ ከ ህህ ፎር 


A DISCUSSION OF ከ ...||ቄ.5.1ከድከዝ.ጊኪከኒጊከ9.495554+ዙሱ5ሉ፡እቹሽ ” ዋከ ነ 


PREISENDORFER (198 ሾ | ኤዚ 116. ብህ... 
ATMOSPHERIC MARINE HORIZONTAL VISIBILITY USING 
MODED OUTPUT STA See. 


Ta INTRODUCTION 


The following discussion is based upon an unpublished 


note by Preisendorfer (1984). The note develops the 


Principal Discriminant Method (PDM) of forecasting and 


suggests how to link the output of numerical weather 


prediction model output parameters with observed fields to 


produce model output statistics (MOS) prediction schemes. 


The application of his methodology to MOS forecasting is as 


follows: 


14 


Generate suitably lagged predictand/predictor 

pairs of data. The predictors are drawn from the 
United States Navy Fleet Numerical Oceanography 
Center's Navy Operational Global Atmospheric 
Prediction System (NOGAPS) model output. The 
predictands are drawn from synoptic ship visibility 
observations provided by the Naval Oceanography 
Command Detachment, Asheville, North Carolina. 


separate the predictand data into visibility 
categories. Construct predictand/predictor pairs 
based on the predictand visibility category values. 
Partition the space of predictor values into 
category subsets. 


Fit a probability density function to the category 
subsets of predictor space. This task is facilitated 
by uSing a succession of principal component analyses 
of the category sets in predictor space. 


Based on the probability density functions for the 


training set, find the potential class errors, PAO; 
PAl. 
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5. Based on the probability density functions and 
utilizing testing set data, find the actual class 
ECroOrsSs AO 1ሬ 

u CSS the first predictor the one with the smallest 
averaged P-value (a measure of separation between 
two probability density functions) and largest PAO 
value. 

Correlate a potential predictor with the set of 
already selected predictors, selecting as the next 
predictor the one which is least correlated with 
the already-selected set. 


8. Repeat steps 1-5 and 7 until all predictors are 
chosen. 


INE BE PREDICTOR sTAGE 


A. THE PREDICTOR/PREDICTAND PAIR 

For each individual data point I (I=1,NTRN, the number 
of points in the training set) there is a predictand value 
NTRPY(I) and its corresponding predictor values TRNPX(I,KX) 
where KX-1,KP, the total number of predictors under 
consideration. For this study, the NTRPY(I)'s represent 
Ability while the TRNPX(I,KX) may be, e.g., the vapor 


pressure at 925 mb, or surface moisture flux, etc. 


Eve tae DISCRIMINANT SET 
The discriminant diagram, Fig. 2a, for the data, shows 


histograms indicating to which predictand category a given 


"The notation herein follows that of the corresponding 
computer code. 


ከ 


predictor value is assigned. Thus the triangles are for 
category 1, circles for category 2, squares for category 3. 
In Fig. 2b, these histograms are separated vertically to 
form the discriminant set. As the points in the data set 
are considered (i.e., as I changes), the TRNPX(I,KX) value 
moves irregularly about on the horizontal axis while the 
corresponding NTRPY(I) moves similarly among the three 
levels of the vertical axis now occupying category 1, the 
category 3, and so on. A point pair is shown in an 
instantaneous position in Fig. 2b. There are NTRN such 
pairs in the dependent (training) discriminant set and NTST 
such pairs in the independent (testing) discriminant set. 
The diagram in Fig. 2b stands for either of these two 


distmiminans sem. 


C. CATEGORY SUBSETS OF PREDICTOR SS ae 

Looking at the discriminant set (Fig. 2b), notice the 
subset of predictor points associated with category one. 
This is XCAT1(II,KX), the r$ehtmost pares 06 pomos On uae 
first level (which is simply a copy of the horizontal axis). 
Similarly, XCAT2(I2,KX) contains the middle pairs and 
XCAT3(13,KX) the leftmost pairs. Each predictor point of 
the training set is assigned to a predictand in a particular 
category. Thus predictors corresponding to the training 
predictand values in categories 1,2 or 3 are assigned to 


XCAT1, XCAT2 or XCAT3 respectively. I1, I2 and I3 represent 
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the index of values in the respective categories. KX 
Gdentifies the predictor (e.g., vapor pressure at 925 mb., 


EXC. ). 


ከ የመው | ከኮ ዜሩ THE PROBABILITY DENSITY FUNCTION 
fer This study, the Gaussian probability density 
function (PDF) was chosen to be fitted to the category 
subsets of the predictor space. However, one might consider 
using other PDF's if they were more suitable for a given 
data set. 
The one dimensional Gaussian PDF for category J is: 
PHIJ=(29)72*(SIGI)~+*EXP(-0.5((X-AVGJ)**2/VARJ) 
where J=1,2,3, and 
AVGJ=average 
SIGJ=standard deviation 
VARJ=variance 
Peete set of points defined by XCATJ(IJ,KX), IJ-1,NXJ for 


each predictor indexed by KX. The fitted curves may appear 


Meet Fig, 2c. 


E. CLASS ERRORS 

An indication of how well a prediction method is doing 
is to count the number of predictions that are correct 
(zero-class errors) and the number of predictions that are 
off by one category (one-class errors). This is done two 
ways. The potential zero-and one-class errors, PAO and PAI, 


are determined using the probability functions fitted to the 
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category subsets of the training set. The actual zero-and 
one-class errors, AO and Al, are determined using the 
testing set. 
1... Findings Whew prebabilitices 
Form the array 
ANU(M,J)=PHIJ(TRNPX(M,KX)), KX fixed, 
where J=1,2,3 
TRNPX is the training set function which assigns 
to (M,KX), a predictor value 
M=1,...,NTRN the indexes of points in the training set 
KX=1,....KP the set of predierocs Aden 
Let 3 
SNU(M)=> ANU(M, J) 
and iius probabilities: 
PRB(M,J)=ANU(M,J)/SNU(M). 
2. Finding TERO RAI 
Find the maximum of the set of probabilities 
PRB(M,J),J=1,3. Let this be PRB(M,J(M)) for each 
M=1,NTRN.For example, if of PRB(M,1), PRB(M,2) and PRB(M,3), 
the maximum value occurs for PRE(M,2), then 2 een 
practice, this would result in predicting category 2. Thea 
define 


1 NTRN 
PAO-grgg 2. PRB(M, J (M) ) 


NTRN 
PAl-gegg-2. CAPRB(M, J (1) * APRB(M,J(M)*2)] 


PA2- 1 - (PAO + PA1) 
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where APRB(M,1)=0 

APRB(M,2)=PRB(M,1) 

APRB(M, 3)=PRB(M, 2) 

APRB(M,4)=PRB(M, 3) 

APRB(M,5)=0 
Tre APRB arrays allow for easier calculation of PA1 since 
array indexing does not allow for PRB(M,O) terms, for 
example. 

The higher the PAO values and the lower the PA1 values, 
the potentially better the predictor PX(I,KX) may predict 
PY(I). Therefore, a potentially good predictand-predictor 
pair has large PAO and small PA1 values. 

SR ling AO, Al 

AO and Al are the actual zero- and one-class errors 
produced by the model when the predictor values of the 
testing set are given to the previously established 
probability density functions, i1.e., the PHIJ's. Using the 
same strategy as for PAO,PA1 make a prediction for the 
predictand value and then compare it with the actual 
predictand value from the testing set. With each correct 
prediction or one-class error, the totals of AO or Al 
increase by one unit, respectively: 

AO=(1/NTST) (TOTAL ZERO-CLASS ERRORS) 


Ai=(1/NTST) (TOTAL ONE-CLASS ERRORS) 
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F, SCREENING AND RANKING CATEGORY SUBSETS 
1. Separability of Category Subsets 

Unless the category subsets are well separated from 
each other, the predictions will not have much skill. As a 
measure of separability, the P-statistic of each distinct 
pair of categories is found for each predictor using BMDP 
Statistical Software program P7D [University of Califor kidi 
1983]. For the three-category case at hand, this provides 
three P-values which are then averaged to provide a single 
mean P-value for each predictor. These are then ranked 
smallest to largest: the smaller the value the better 
separated are the data heaps in the category subsets. The 
first chosen predictor is thus the one with the smallest 
mean P-value. (Other measures of separability exist. See, 
e.g., the potential predictability (PP) measure in 
Preisendorfer (1983a). 

2. PAO scores 

In the event that more than one predictor has the 
smallest mean P-value, then the first predictor is chosen by 
selecting from among those predictors the one with the 


largest PAO score. 


“Unless category set separation is present, it is 
unlikely that any method of prediction of the given 
predictand from the given predictors will be skillful. It 
is this feature of the prediction problem that discriminant 
methods, such as the present one, isolate most clearly: if 
category probability density curves are well separated and 
the training set is representative of the data set, then 
high forecast skill is assured. 
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ieee LC TOR. STAGE 


A. CORRELATIONAL SCREENING OF PREDICTORS 

Suppose we have K-1 predictors selected, where K-2,..,KP 
and KP is the total number of predictors. Let the selected 
predictors be TRNPX(I,KX), KXz1,..,K-1. So, if there is one 
chosen predictor we have TRNPX(I,1), I=1,NTRN. Let the 
remaining set of predictors be denoted TRNPW(I,KW), 
KW=1,...,L where L=KP + 1 - K. Let CORR[KW,KX] denote the 
correlation between the KXth chosen predictor and the KWth 
unchosen predictor. Since the correlation is a measure of 
the distance between a chosen and an unchosen predictor, we 
are looking for the smallest value of the correlation since 
a smaller value indicates that the unchosen predictor is 
farther from (i.e., less dependent on) a chosen predictor. 

Therefore, let 

C(KW)=max {ABS CORR [KW,KX]J 

This gives (an inverse) measure of the distance of the KWth 
potential predictor from the set of chosen predictors. The 
smaller C(KW) is, the larger the distance. The next 
predictor chosen for consideration is the one with the 


smallest C(KW) value. 
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B. THE K-DIMENSIONAL DISCRIMINANT SET 

Once the Kth predictor is added, there are two sets, one 
training and one testing, of K predictors in thewPoOPM ONES 
Vektor 

VEGPX ( 2) St Pe rere MN 
in the Euclidean K-space E. As index I changes, VECPX(I) 
moves about in E as does the predictand array NTRPY(1). 
The set of all ordered pairs (VECPX(I),NTRPY(I)), I-1,NTRN 
is the present discriminant training set and is a general 
(k-stage) version of section II.B. above. 
(VECTS(I),NTSPY(2)),2=1,NTST is the discriminant Les ón 


set. 


C. CATEGORY SUBSETS OF PREDICTOR PA E 
As in II.C. above, category subsets of the K-dimensional 
predictor training vector are formed, based on the value of 
the associated predictand value. The net result is three 
subsets of E, defined by the three swarms of points 
ÍXCATJ(1):  I=1,NXJÍ 
where J=1,2,3, and NXJ is the number of points in the 
subset. Here XCATJ(I)=[XCATI(1,1),...,XCATJ(K,1)]%. On 
these three subsets of E. we fit the K-dimensional Gaussian 


PDF's. However, these data swarms are not usually 


distributed normally which brings about the next step. 


De 


IB INARY PRINCIPAL DECOMPOSITIÓN OF THE CATEGORY SUBSETS 
Let X = {XCATJ (I): ከጠ ረ ፡፤.2,..:06 the Jth 
category subset of 2 A general picture of A for the case 
ከ ።]ክ Fig. 3. The shape of X í may possibly be elongated 

and curvilinear. Since the most variance of the subset is 


ome tne unit vector e, at 0, this suggests that we form a 


1 
principal component decomposition of the swarm A, of NXJ 
points in E. where J=1,2,3. The principal component 
decomposition is well-suited to find this direction 6) oí 
greatest variance. This is done in the following steps. 
e Recall that in III.C., the data sets forming the 
predictors were standardized. Next, go on to fina the 


peneewold AMEANJ of each XCATJ in Er This is the centroid 


pL shown as O in Fig. 3. By definition, 


-<=' 


1=1 
The Lth component of AMEANJ is 


EX 
AMEANJ NXJ XCATJ(I) 


AMEANJ(L)-—2  XCATJ(I,L) 
ይ ጋዜ 
here L=1,..., K. 
በ09 119 .ክር covariance matrix SJ of the Á í data swarm. 


Thus, first center the points XCATJ(I) on the mean AMEANJ of 


i Ge) =XCATI( 1) = AMEANJ, I=1,...,NXJ 
ከ in component form 

L x APJ) AMEANJ (L) 

Ay መ «| «ብ K 


Then the entry SJ(L,M) of SJ in its Lth row and Mth column 
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18" 


NXJ 
= ss 
SI(L,M)= 35 2L) M) 
for L,M-1,...,K and for categories J IP: 


3. Find the eigenvalues and eigenvectors of the 
covariance matrix SJ(L,M). Sort the eigenvalues from high to 
low, and arrange their corresponding eigenvectors similarly. 

4, Compute A(I), the principal components from 

A(3)=9 XI(1,L)e, (L) 
where e =le (1),...,0, (8፪7)7" is the eigenvector 
corresponding to the largest eigenvalue of the data swarm 
currently under consideration. 

The following steps describe how the above information 
is used in decomposing the data swarms to a terminal state 
for use in the multi-predictor PDF's. See Fig. 3 for Teas 
O of the splitting procedure. 


5. Decision to split subsets at level 1: 


a. For K predictors and a set X,(a,,...,ag) of ny 
points: 
ከ n S K+1, where K=number chosen predictors, 
set ፲7(ወ1›**፣፣ወሪ) = X (a y. .., 47). This set is terminal 


because any further splits of the set will lead to 
degeneracy. In fact, if "Dech, set PHIJ=0 for this set since 
in this case only trivial covariance matrices will be found. 
POS ny> K+1, Zo toro: 
b. Perform principal component analysis (PCA) of the 


point swarm Xy lay seses ag) in E + Determine the eigenvalues 
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EU es iO is the largest and ያ is the smallest. 


k 
Let ሪ=ቃ ያ, Compute AE > £,). Goero ር. 
1 


i 
=] 
c. Perform a Monte Carlo experiment to determine if A 


is significantly large. That is, randomly generate a 
duplicate of the data swarm under consideration, normalize 
the data and find the centroid, covariance matrix and 
eigenvalues. Let # 272.,.2. be the ordered set of 
eigenvalues resulting from the ith Monte Carlo experiment, 
95,100. 1652. = ye and set Mi)35£/(4=£ ). 
Arrange the A(i) in ጆች order, so that after 
relabeling, A(1)<A(2)S ...<A(96)<...A(100) , 

| TT £,=0, or ny jJ Í set PHIJ=0. 

PT; Set Tjlajreersag)=K la ,....a,). 
This is the terminal case. Go to the next swarm awaiting 
composition. | 

3) If A(96)<A, go to d. 

d. A split is performed by setting 


X (a, see, a,={X(I): X(1)€ X,(2a,,...,a,) and A ens 


1 
Ste, = EERE > 
X (a, a,)*iX(1): X(I)EX (a, az) and A,(I)>0] 
where X(I) is a point (k-tuple) of numbers in Er When 
splits are completed on all levels, we have a set of 
BeErminanodes Pla, 1... 107). See Fig. 4. 


NCG PROBABILITY DENSITY FUNCTIONS TO EACH TERMINAL 
NODE 


Denote the terminal nodes by ከከ ከ which is the name 


of the Ith terminal node found by successive splits of Xy in 


ED 


e (Note: if the terminal node results from a degeneracy 
or from the case ና then no further work is done since 
PHIJ=0 in those cases for all points.) Establishing the 
following notation: 

AVGJ(I)=centroid of the terminal swarm of points TU) 

€ (I)-KxK covariance matrix of T) 

DET (I)ezdeterminant on C (I) where 


k 
DET (I)s/I2 , £ » eigenvalues of C (I). 
r=1 Y E E 


H 
Then the required probability EE EE E 


-1 


; (1)(X-AVGJ(1))] 


gue 1 
PHIJ(1,X)=[2m]*[ DET ,(T).] **ጅ፳ጅ[-0. 5* (፪-ልሃ67(፲)) ር 


where I runs over all terminal nodes associated with 
category subset X í 
X is an arbitrary point in E, 
X - AVGJ(I) is a k-component column vector in E 
'T' denoting transpose 
pM is the inverse of the covariance matrix e (1). 
This results in a set of three probability distributions 


PHIJ(I,X), J=1,2,3 and forms the present model over each Xo 


after suitably assembling these PHIJ(I,X) values. 


F. ASSEMBLING THE PHIJ(I,X) ON EACH X 


Let n,(I) be the number of points in T (1) Then 


M 
Yn 
l=] 


where M is the number of terminal nodes arising in X. 


y 
sit) anxJ, the number of points in Gs 


Define 
a, (I)=n,(1)/NXJ. 
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= 
C€ 


— 


=1 


M 
J 
Then 2,2 (1)71. Set PHIJ(X)- 5 a (I)PHIJ(I,X) 
1=1 
ይ ከ1, 2,3, X in SIS This is the desired model. 


ERU CLASS ERRORS 
These are made from the new versions of PRB(M,J(M)) 


computed as in section II.E. above. 


H. FINAL SCREENING TESTS FOR CANDIDATE PREDICTOR PX(I,K) 

1. Using BMDP program P3D compute the P-value for each 
of the three possible pairs of PDF's for the three 
@aeesories. Average these values and find P. 

2. Compare the new PAO and PAl values with those found 
for the previous run with one less predictor. 

3. Compare the new PAO to the null hypothesis. 

“Accept PX(I,K), the Kth candidate predictor, if each of 
the following hold: 
:.ከክርሟውቄ 
b. PAO(K-1)«PAO(K), PA1(K)SPA1(K-1) 
c. . PAO»PAO(nu11) 
Here PAO(null) is the upper limit of the 95% confidence 
Bpubterval, as found in Appendix B. 

If these conditions are not fully met, return to section 
MULA. and select the next potential predictor PW(I,KW) in 


line, until all potential predictors have been considered. 


“In the original version of PDM [Preisendorfer, 1984], 
this step uses the potential predictability (PP) criterion. 


p 


Once the model is finished, that is all potential 
predictors have been considered, then compute the actual 


AO(I) and A1(I) scores using the testing set. 


19, EXAMPLE 


An example is helpful in understanding how the method 
works in practice. The results presented here were obtained 
by applying the method to a set of 200 points taken from the 
Area 2, TAU-24 data set. The example will extend through 
one level of the multi-predictor stage, i.e., through the 


selection and acceptance of a second potential predictor. 


A. SELECTION ORT Ie DEA DR 

The first step towards identifying the first predictor 
ls to run the BMDP Statistical Softwares oare © 
[University of California, 1983], to find the average 
P-value for each predictor. In this case, there were 
several predictors for which the average P-value was 0.0. 
Therefore, (see note 2 of II.F.) the results showing the 
PAO, PAi, AO and Al scores for each predictor (if used as 
the first predictor) had to be consulted before the choice 
was made. The chosen predictor was E850 because of all the 
predictors with an average P-value of 0.0, it had the 


largest PAO score (i DR 
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Predictor E850 was then correlated with the remaining 
potential predictors. The potential second predictor chosen 
was DEDP because it had the smallest correlation coefficient 


when correlated with E850. 


THE SECOND PREDICTOR STAGE 
With the first predictor chosen and a candidate second 
predictor ready for consideration, it was necessary to begin 
the principal component analysis (PCA) of the data swarm in 
wee i pation of creating the probability density functions. 
When broken into the three categories corresponding to 
the visibility groupings, the categorized data sets 


contained the following numbers of points: 


XCAT1--14 
XCAT2--14 
XCAT3--172 


The decomposition of the first category subset (XCAT1) 
will be explained in detail, since it is small. Fig. 5 
presents a pictorial representation of the following steps 
in the K=2 (predictor) stage: 

one ldes ስለሰጠ first. For this swarm, A>A(96). 
Therefore, the swarm must be split using PCA. The two new 
pets are X, (0) with 6 points and X,(1) Atna S points: 

REEL Consider the swarm X, (0). once coal En tuus 
meee tie set is terminal, i.e., T, =X, (0). Ne ts Urther 


decomposition is performed on this set. 
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3. Consider the swarm X,(1). Here, A>A(96), so the swarm 
is further decomposed into X,(10) with 5 points and X,(11) 
WESSEN 

4, Next X, (10) is considered. Since A>A(96), this swarm 
is further decomposed into X,(100) with 4 points and X,(101) 
with on 

5. Next X,(11) is considered. Since this swarm n ጋ 
3 values, and 3 K-*1, this swarm is terminal. Thus, 
TA 

6. The data swarm X, (100) with 4 points is found to have 
A>A(96) and therefore, it is terminal. Set T4-X,(100). 

7. These XQ LOU has only 1 point. Since IPK P DE 
set is degenerate. Although DKE DE for this terminal 
set PHIJ=0 for all values of X. Therefore, it is not 
considered when building the probability density functions. 

Thus for XCAT1, there are three useable terminal sets. 
Similarly, there are two for XCAT2 and fourteen for XCAT3. 

Once the PHIJ's are formed and probabilities computed, 
potential class errors are computed and compared to the 
potential errors found at the one predictor level. The new 
PAO (.67) is greater than at the one-predictor level (.51) 
and the new PA1 (.27) is lower than at the one predictor 
level (.39). With part of the selection criteria satisfied, 
the average P-value using both predictors was found using 
BMDP Statistical Software program P3D [University of 


California, 1983]. Since the average P-value (0) met the 
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Significance criteria of being less than .05, a second 
requirement towards acceptance of the second predictor was 
met. Since PAO (.67) was greater than PAO(null)=.40, the 


third criteria was met and the second predictor DEDP was 


accepted. 
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APRENDI 1. 
NULL HYPOTHESIS SIGNIFICANCE TESTING 


Following the work of Diunizio (1984a), Mr. Paul Lowe of 
NEPRF proposed that statistics such as AO and threat scores 
could be assigned normal probability distributions and, 
therefore, be subject to Null Hypothesis significance 
testing criteria. The assignment of the normal probability 
distributions is based upon the Central Limit Theorem. 
Diunizio (1984b) explored this technique and presented the 
subsequent results. This appendix presents the equations 
used in this study for significance testing. 

When using three visibility categories, the null 
hypothesis is that the percentage correct will be .333 if 
only chance is involved. Using a 95% confidence test, we 
want to create an interval around the null hypothesis value 
such that values outside are considered to be significant. 

Let Po=.333 

n=number of values in data set 
Z ajo = 1.96 for 95% confidence interval 
(1 -a 7.955 +. a/2=.025) 
then AA=Po - za LPo (1 - Po)/n] 

2፡0 ER Eech 

where AA is the lower limit and BB is the upper limit of the 


confidence interval. 


62 


APPENDIX C 


WORLD METEOROLOGICAL ORGANIZATION 
HORIZONTAL SURFACE VISIBILITY CODES 


CODE VISIBILITY (KM) 

90 <0 O5 

91 0805 

92 Ë 

2 Dk 

94 Ein 

93 250 

96 ge 

| 10.0 

98 ከ 

oo 50.0 or more 

Note: The values given are discrete values (i.e., not 
ranges). If the observed visibility is between two 


reportable distances as given in the table, the code figure 
of the lower reportable distance shall reported. 
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APPENDIX 


SKILL AND THREAT SCORES, DEFINITIONS (Karl, 1984) 


FORECAST 





1 2 3 
OBSERVED 


Total = Rosh oS See Bae ree pap o p Z 


P1 (T+W+Z)/Total 


(ክ.ከ ፣ Oy taro P3 


P2 (S+V+Y)/Total PN = greatest of P1, P2 or P3 


Raw Scores 


AO = fraction correct = zero-class error e OVE CEN 
Al = one-class error = (U+S+Y+W)/Total 
ለ2 = two-class error = (R+Z)/ Toral 


AO + Al +42 2m 
TS1 = Threat score Tor visible comal 


X/(R+U+X+Y+Z) 


152 Threat score for visibility category II 


V/(S+V+Y+U+W) 
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K = Threat score for visibility categories I and II 


(X+V)/(Total-T) 

TS12 is designed to represent the skill of forecasting 
Visibility categories I and II as separate categories, 
rather than their skill as a combined category, which would 


be (U+V+X+Y)/Total-T). 


Adjusted scores 


PAG = (AO-PN)/(1-PN) 


Bue) (TS1-P1)/(1-P1) 


(TS2-P2)/(1-P2) 


ATS2 
ATS12 = (TS12-(P1+P2))/(1-(P1+P2)) 
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tes 


NOGAPS PREDICTOR PARAMETERS AVAILABLE FOR 


APPENDIX E 


NORTH ATLANTIC OCEAN PAPER F F 


Area: North Atlantic Ocean and Mediterranean Sea 


Model output time: 


Legend: * 


Model output 
parameter 


D1000 
D925 
D850 
D700 
D500 
Don * 
D300 + 
D250 * 
TAIR 


T1000 


Parameters which were not used because they 
were considered physically unrelated 


1200GMT (TAU-00, TAU-24, TAU-48) 


15 May--7 July 1983 


tó marine visibility: 


Parameters which were not used due to loss 
of significant digits during transter sige 


tape to mass Storage. 


Parameters existing for TAU-24 and TAU-48 


only. 


Descriptive name of parameter 


1000 mb geopotential height 


eÐ 
850 
700 
500 
400 
300 


250 


mb 
mb 
mb 
mb 
mb 
mb 


mb 


geopotential height 
geopotential height 
geopotential height 
geopotential height 
geopotential height 
geopotential height 


geopotential height 


surface air temperature 


1000 mb temperature 


66 


T925 
1700 


T 500 


LOO ~“ 
j OO 
| 0 > 


EAIR 


E1000 


E925 
E850 
E700 
E500 


UBLW 


U1000 


U925 
U850 
፲700 


T300 


፲:00 ! 
፲300 * 


U250 : 


VBLW 


V1000 


V925 


925 mb temperature 

ኔታ” ጅ ያጅዜ፡ ure 

500 mb temperature 

LOO mb temperature 

300 mb temperature 

250 mb temperature 

Surface vapor pressure 

ከ የ ከ ከ ከ ቨ ዘጠን». ። ክሉ! ጌ ከ ከ [ያ፦ር 

AO MM Ao pressure 

850 mb vapor pressure 

(OIM Vapor pressure 

s iS po pressure 

Boundary layer zonal wind component 
1000 mb zonal wind component 
925 mb zonal wind component 
850 mb zonal wind component 
700 mb zonal wind component 
500 mb zonal wind component 
400 mb zonal wind component 
300 mb zonal wind component 
250 mb zonal wind component 
Boundary layer meridional wind 
component 

1000 mb meridional wind component 


925 mb meridional wind component 
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V850 

V700 

V500 

v400 * 
V300 ~ 
V250 * 
VORJ2 5 5 
MORSU 
r. 

SMF 

251818 

፡ 1.11) 
STRTTH 
SE 


ENTRN 


DRAG ** 


PRECIP gz 


INSTAB *** 


DIV925 3t 36 46 


850 mb meridional wind component 

700 mb meridional wind component 

500 mb meridional wind component 

400 mb meridional wind component 

300 mb meridional wind component 

250 mb meridional wind component 

925 mb- vorticity 

SOLD Vortice ity 

surface pressure 

Surface moisture flux 

Planetary boundary-layer depth 
Percent stratus frequency 

stratus thickness 

surface heat flux 

Entrainment at top of marine 
boundary-layer 

Drag coefficient (C p) 

Total amount (mm) of model 
precipitation in the last six hours 
Total amount (mm) of model precipita- 
tion associated with cumulus convection 
in the last six hours 

Boundary layer inversion instability 


925 mb divergence 
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Derived Parameters 


DIDP 


DEDP 


DUDE? 


DVDP 


RH 


TA 


DDVDP 


EENS + 


DUUPDP 


ን ከ ከሠ!) 


ESUM 


EPRD 


HDIF 


Vertical gradient of temperature 
(1000-925 mb) 

Vertical gradient of vapor pressure 
(1000-850 mb) 

Eeer engt, of zonal wind 
(1000-850 mb) 

Vertical gradient of meridional wind 
(1000-850 mb) 

Surface relative humidity 

Virtual temperature 

Vertical gradient of geopotential 
height (1000-850 mb) 

Peuuiecal oradscmusot vorticity 
(500-925 mb) 

Vertical gradient of zonal wind 
(300-500 mb) 

Vertical gradient of meridional wind 
(300-500 mb) 

oum of vapor pressures 

(1000 & 850 mb) 

Product of vapor pressures 

(1000 & 850 mb) 

Difference of vapor pressures 


(1000-850 mb) 
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(a) 


(b) 


cc) 


Pig. 


Frequency of 
Occurrence 


NTRPY(I) 
N 


Frequency of 
Occurrence 


po 


DATA TRAINING SET 


0 A 
8 ወ * ል a a ሲል Category 1 
s [3 Se 806A BeA ወል ል ል E Category 2 


e - Category 3 





ብ! 1 እኣ), fixed KX 


DISCRIMINANT TRAINING SET 


C7 ORNEXCI, KX ENIBRSPY CES 


e 9 ° KX fixed, I given point 


URNPXQMEKX), fixed KX 


POSSIBLE FITTED PDF'S 
Categories 





Distribution diagrams for a sample training set 
where (a) shows the vertical stacking of 
Spservavicns; (bp!) shows the (a) data in their 
category discriminant sets; (c) shows the analytic 
representation of the data in (a). 


Ce 








X 01) 
ዕ (level 1) 


X, (0) 
(level 1) 









J 
(level O) 


2nd PREDICTOR AXIS 


Ist PREDICTOR AXIS 


Fig. 3. A general representation of X; in the case of 
two predictors, from Preisendorfer (1984). 
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XCAT1 
14 PTS 


6 PTS 8 PTS 


X4(10) X4( 11) 
22፡7) 3 PTS 


Terminal -To 


X (100) X4 (101) 
4 PTS 1 PT 


Terminal-T4 Terminal -Ta 


Terminal - Ty 


Fig. 5. Schematic of the binary decomposition of a sample 
set from area uae. 
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Fig. 7. Comparison of PAO scores for FATJUNE 1993, Dr 
Atlantic Ocean area 2(A,B,C), TAU-24, PDM model. 
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Fig. 9. Comparison of IND AO scores for FATJUNE 1983, Nore 
Atlantic Ocean area 2(A,B,C), TAU-24, PDM model. 
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Fig. 13. Comparison of PAO scores for FATJUNE 1983, North 
Atlantic Ocean area 2(A), TAU-24, PDM model 
criteria tests. 
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Fig. 19. Comparison of PAO scores for FATJUNE 1983, Norah 
Atlantic Ocean area 2, TAU-24, Case Y(A,B,C), 
PDM model. 
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Fig. 21. Comparison of IND AO scores for FATJUNE 1983, North 
Atlantic Ocean area 2, TAU-24, Case Y(A,B,C), 
PDM model. 
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